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DETAILED ACTION 



Response to Arguments 



Applicant's arguments, see page 12, filed May 6, 2004, with respect to the prior 
art references failing to disclose applying a bilinear transformation to selected pixels 
have been fully considered and are persuasive. The previous rejection has been 
withdrawn. 

Because of the Examiner failed to meet all the limitations of the claims in the 
previous office action, the following action will be Non-Final. 



The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth In section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

1 . Claims 1 , 4-6, 1 0, 1 3-1 6 and 31 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Szeliski et al (US6044181) in view of Okitsu (US61 88800). 
Regarding claims 1 and 31, Szeliski discloses a method comprising: 
synchronously capturing a set of images from a camera array (see col. 8, line 49 
to col. 9, line 58); 

selecting pixels from at least one image of the set of images (see col. 1 1 , lines 1- 



Claim Rejections - 35 USC § 103 



12); 
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applying a planar transformation to the selected pixels to transform the selected 
pixels from a coordinate system of the at least one image to a common coordinate 
system of a composite image (see col. 10. line 9, line 59 to col. 1 1 , line 1 1 ; Szeliski 
teaches combining two images taken at different viewpoints and computing a planar 
perspective transformation between each overlapping pair of the images). 

Szeliski does not expressly disclose applying a bilinear transformation to the 
selected pixels. Okitsu teaches a two-dimensional spatial transformation system. 
Okitsu reveals, in figure 24A-D, that it is well known in the art to apply a bilinear 
transformation to overlapping portions of images to produce a composite image (see 
col. 20, line 5 to col. 21 , line 21 ). It would have been obvious to one of ordinary skill in 
the art at the time the invention was made to modify Szeliski's device by implementing 
Okitsu's teachings. One would have been motivated to do so in an effort to produce a 
composite image with a simple hardware configuration and reduced cost. 

Regarding claim 4, Szeliski discloses identifying a set of contiguous patches in 
each of the images (see col. 17, lines 10-46). 

Regarding claim 5, Szeliski discloses identifying regions in each image having 
registration points in common with the composite image (see col. 20, lines 34-60). 

Regarding claim 6, Szeliski discloses combining patches from different images of 
the set of images having a same set of registration points into a same corresponding 
location of the composite image (see col. 24, lines 1 1-44). 

Regarding claim 10, the combination of Szeliski and Okitsu disclose applying a 
predetermined bilinear transformation matrix to each patch, the predetermined bilinear 
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transformation matrix representing an amount of warping required for each patch to 
transform each patch into the common coordinate (see col. 11, lines 1-12; also see col. 
8, lines 49-67). 

Regarding claim 13, Szeliski discloses repeating the step of applying a 
transformative equation for each set of images synchronously captured by the camera 
array, each set of images representing one frame in a video stream of the scene (see 
col. 10, lines 2-6). 

Regarding claim 14, Szeliski discloses selecting an area of interest from the 
composite image; and outputting the selected area of interest to a user (see figure 18). 

Regarding claim 15, Szeliski discloses the step of outputting comprises the step 
of displaying the area of interest from the combined warped images (see figure 2B). 

Regarding claim 16, Szeliski discloses directing the area of interest to a 
predetermined area surrounding the proximity of objects detected in the scene (the 
examiner notes that Szeliski "area of interest" is the overlapping region). 
2. Claims 7 and 8 are rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Szeliski et al and Okitsu in view of Harding (US5966177). 

Regarding claim 7, Szeliski discloses warping patches from different images 
having a same set of registration points into a same corresponding location of the 
composite image. Neither Szeliski nor Okitsu expressly disclose cross-fading patches. 
Harding teaches an image display apparatus capable of blending multiple single display 
images together. Harding further teaches that it is well known in the art to correct 
intensity in an overlapped region of display images using a fading technique (see col. 3, 
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lines 1-29; also see col. 4, lines 41-65). It would have been obvious to one of ordinary 
skill in the art at the time the invention was made to implement Harding's teachings. 
One would have been motivated to do so in an effort to display image data of the 
correct intensity in an overlapped region. 

Regarding claim 8, see claim 7 above. 
3. Claims 20 and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Henley (US5657073) in view of Szeliski et al in further view of Okitsu. 

Regarding claim 20, Henly discloses, in figure 3, a camera array comprising: 

a set of cameras mounted In an array (10; see col. 4, lines 22-33); 

an image combining mechanism (16) configured to combine at least two of 
images captured from the set of cameras into a composite image (see col. 4, lines 34- 
57), the combining mechanism including: 

a view selection device (20, 22) configured to select a view from the composite 
image (see col. 4, lines 34-57); and 

an output mechanism (24) configured to display (30) the selected view. 

Henley does not expressly disclose a warping device configured to warp patches 
by applying a bilinear transformation. Szeliski discloses warping patches from different 
images having a same set of registration points into a same corresponding location of 
the composite image. Szeliski does not expressly disclose applying a bilinear 
transformation. Okitsu teaches a two-dimensional spatial transformation system. 
Okitsu reveals. In figure 24A-D, that It is well known in the art to apply a bilinear 
transfonnation to overlapping portions of images to produce a composite Image (see 
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col. 20, line 5 to col. 21 , line 21 ). It would have been obvious to one of ordinary skill in 
the art at the time the invention was made to modify Szeliski's device by implementing 
Szeliski and Okitsu's teachings. One would have been motivated to do so in an effort to 
produce a composite image with a simple hardware configuration and reduced cost. 

Regarding claim 22, Henley discloses the frame selection device includes a 
stereoscopic ranging mechanism configured to utilize at least two images from separate 
cameras of the camera array to determine a range of objects in the composite image 
(see col. 4, line 61 to col. 5, line 12; the Examiner notes that Applicant's claim language 
is written in the alternative). 

4. Claims 21 , 23 and 24 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Henley and Szeliski and Okitsu in view of Harding. 

Regarding claim 21, Henley discloses combining image data from multiple 
cameras. Neither Henley nor Szeliski nor Okitsu expressly disclose a fading device. 
Harding teaches an image display apparatus capable of blending multiple single display 
images together. Harding further teaches that it Is well known in the art to correct 
intensity in an overlapped region of display images using a fading technique (see col. 3, 
lines 1-29; also see col. 4, lines 41-65). It would have been obvious to one of ordinary 
skill in the art at the time the invention was made to implement Harding's teachings. 
One would have been motivated to do so in an effort to display image data of the 
correct intensity in an overlapped region. 

Regarding claim 23, the combination of Henley and Szeliski disclose the array of 
cameras are immovably mounted on a firm base (housing 12 in figure 2 of Henley); and 
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the warping device applies a pre-deternnined transformation to each of the patches 
(Szeliski teaches a planar transformation). 

Regarding claim 24, the combination of Henley and Szeliski disclose a 
registration mechanism configured to register each of the camera arrays by finding 
registration points in common with views of each camera and the composite image 
(Szeliski teaches finding registration points only in overlapping regions of combined 
images). 

Allowable Subject Matter 

1 . Claims 28-30 are allowed. 

Regarding claim 30, the prior art does not teach or fairly suggest a method of 
registering a camera array comprising placing at least one registration point in a field of 
view of at least two cameras, identifying a location of each registration point and 
maintaining information about each registration point, wherein 

the step of maintaining information comprises calculating a warped coordinate 
system and maintaining a table identifying pixels of the cameras, an amount of fade, 
and a corresponding location in the warped coordinate system. 

2. Claims 1 1, 12, 25 and 26 are objected to as being dependent upon a rejected 
base claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

Regarding claim 1 1 , the prior art does not teach or fairly suggest a method of 
warping a set of images into a common coordinate system of a composite image by 
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cross-fading patches from different images having a same set of registration points, 
wherein 

the step of cross-fading comprises the steps of varying a parameter of each pixel 
from a minimum value a first and second patch to a maximum value of the first and 
second patch, respectively, summing each of corresponding pixel values of each first 
and second patch and placing the summed values in corresponding locations of the 
common coordinate system. 

Regarding claim 25, the prior art does not teach or fairly suggest a camera array 
comprising a set of cameras mounted in an array, an image combining mechanism, a 
view selection device, an output mechanism and a registration mechanism, wherein 

the registration mechanism comprises a registration point source and a detection 
mechanism. 

Conclusion 

1 . The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. Aiger et al teach a method of mosaicing using affine or bilinear 
transformation. Nakaya et al teach a synthesizing method using bilinear transformation. 

2. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Rashawn N Tillery whose telephone number is 703-305- 
0627. The examiner can normally be reached on 9AM-6:30PM. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Wendy Garber can be reached on 703-305-4929. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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